Variational Analysis of Constrained M-Estimators

We propose a unified framework for establishing existence of nonparametric M-estimators, computing the corresponding estimates, and proving their strong consistency when the class of functions is exceptionally rich. In particular, the framework addresses situations where the class of functions is complex involving information and assumptions about shape, pointwise bounds, location of modes, height at modes, location of level-sets, values of moments, size of subgradients, continuity, distance to a "prior" function, multivariate total positivity, and any combination of the above. The class might be engineered to perform well in a specific setting even in the presence of little data. The framework views the class of functions as a subset of a particular metric space of upper semicontinuous functions under the Attouch-Wets distance. In addition to allowing a systematic treatment of numerous M-estimators, the framework yields consistency of plug-in estimators of modes of densities, maximizers of regression functions, level-sets of classifiers, and related quantities, and also enables computation by means of approximating parametric classes. We establish consistency through a one-sided law of large numbers, here extended to sieves, that relaxes assumptions of uniform laws, while ensuring global approximations even under model misspecification

Log-Concave Duality in Estimation and Control

In this paper we generalize the estimation-control duality that exists in the linear-quadratic-Gaussian setting. We extend this duality to maximum a posteriori estimation of the system's state, where the measurement and dynamical system noise are independent log-concave random variables. More generally, we show that a problem which induces a convex penalty on noise terms will have a dual control problem. We provide conditions for strong duality to hold, and then prove relaxed conditions for the piecewise linear-quadratic case. The results have applications in estimation problems with nonsmooth densities, such as log-concave maximum likelihood densities. We conclude with an example reconstructing optimal estimates from solutions to the dual control problem, which has implications for sharing solution methods between the two types of problems

Solving equilibrium problems in economies with financial markets, home production, and retention

We propose a new methodology to compute equilibria for general equilibrium problems on exchange economies with real financial markets, home-production, and retention. We demonstrate that equilibrium prices can be determined by solving a related maxinf-optimization problem. We incorporate the non-arbitrage condition for financial markets into the equilibrium formulation and establish the equivalence between solutions to both problems. This reduces the complexity of the original by eliminating the need to directly compute financial contract prices, allowing us to calculate equilibria even in cases of incomplete financial markets. We also introduce a Walrasian bifunction that captures the imbalances and show that maxinf-points of this function correspond to equilibrium points. Moreover, we demonstrate that every equilibrium point can be approximated by a limit of maxinf points for a family of perturbed problems, by relying on the notion of lopsided convergence. Finally, we propose an augmented Walrasian algorithm and present numerical examples to illustrate the effectiveness of this approach. Our methodology allows for efficient calculation of equilibria in a variety of exchange economies and has potential applications in finance and economics

Fusion of Hard and Soft Information in Nonparametric Density Estimation

This article discusses univariate density estimation in situations when the sample (hard information) is supplemented by “soft” information about the random phenomenon. These situations arise broadly in operations research and management science where practical and computational reasons severely limit the sample size, but problem structure and past experiences could be brought in. In particular, density estimation is needed for generation of input densities to simulation and stochastic optimization models, in analysis of simulation output, and when instantiating probability models. We adopt a constrained maximum likelihood estimator that incorporates any, possibly random, soft information through an arbitrary collection of constraints. We illustrate the breadth of possibilities by discussing soft information about shape, support, continuity, smoothness, slope, location of modes, symmetry, density values, neighborhood of known density, moments, and distribution functions. The maximization takes place over spaces of extended real-valued semicontinuous functions and therefore allows us to consider essentially any conceivable density as well as convenient exponential transformations. The infinite dimensionality of the optimization problem is overcome by approximating splines tailored to these spaces. To facilitate the treatment of small samples, the construction of these splines is decoupled from the sample. We discuss existence and uniqueness of the estimator, examine consistency under increasing hard and soft information, and give rates of convergence. Numerical examples illustrate the value of soft information, the ability to generate a family of diverse densities, and the effect of misspecification of soft information.U.S. Army Research Laboratory and the U.S. Army Research Office grant 00101-80683U.S. Army Research Laboratory and the U.S. Army Research Office grant W911NF-10-1-0246U.S. Army Research Laboratory and the U.S. Army Research Office grant W911NF-12-1-0273U.S. Army Research Laboratory and the U.S. Army Research Office grant 00101-80683U.S. Army Research Laboratory and the U.S. Army Research Office grant W911NF-10-1-0246U.S. Army Research Laboratory and the U.S. Army Research Office grant W911NF-12-1-027

Sublinear upper bounds for stochastic programs with recourse

Separable sublinear functions are used to provide upper bounds on the recourse function of a stochastic program. The resulting problem's objective involves the inf-convolution of convex functions. A dual of this problem is formulated to obtain an implementable procedure to calculate the bound. Function evaluations for the resulting convex program only require a small number of single integrations in contrast with previous upper bounds that require a number of function evaluations that grows exponentially in the number of random variables. The sublinear bound can often be used when other suggested upper bounds are intractable. Computational results indicate that the sublinear approximation provides good, efficient bounds on the stochastic program objective value.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/47918/1/10107_2005_Article_BF01582286.pd

Challenges in Stochastic Programming

Remarkable progress has been made in the development of algorithmic procedures and the availability of software for stochastic programming problems. However, some fundamental questions have remained unexplored. This paper identifies the more challenging open questions in the field of stochastic programming. Some are purely technical in nature, but many also go to the foundations of designing models for decision making under uncertainty. Key words: stochastic programming, decisions under uncertainty, chance-constraints, probabilistic constraints, distribution problem, Markowitz portfolio model iii iv Challenges in Stochastic Programming Roger J.-B.Wets Recent work in stochastic programming has mostly been aimed at the design of solution procedures and the development of accompanying software; an overly brief review of the present state-of-the-art is provided in x1. This effort should be continued and expanded, and should remain the central concern of the research in stochastic pr..

Stability of ε-approximate solutions to convex stochastic programs

An analysis of convex stochastic programs is provided if the underlying proba-bility distribution is subjected to (small) perturbations. It is shown, in particular,that ε-approximate solution sets of convex stochastic programs behave Lipschitzcontinuous with respect to certain distances of probability distributions that aregenerated by the relevant integrands. It is shown that these results apply tolinear two-stage stochastic programs with random recourse. Consequences arediscussed on associating Fortet-Mourier metrics to two-stage models and on theasymptotic behavior of empirical estimates of such models, respectively

Multivariate Epi-Splines and Evolving Function Identification Problems

Includes erratumThe broad class of extended real-valued lower semicontinuous (lsc) functions on IRn captures nearly all functions of practical importance in equation solving, variational problems, fitting, and estimation. The paper develops piecewise polynomial functions, called epi-splines, that approximate any lsc function to an arbitrary level of accuracy. Epi-splines provide the foundation for the solution of a rich class of function identification problems that incorporate general constraints on the function to be identified including those derived from information about smoothness, shape, proximity to other functions, and so on. As such extrinsic information as well as observed function and subgradient values often evolve in applications, we establish conditions under which the computed epi-splines converge to the function we seek to identify. Numerical examples in response surface building and probability density estimation illustrate the framework.U. S. Army Research Laboratory and the U. S. Army Research Office grant 00101-80683U. S. Army Research Laboratory and the U. S. Army Research Office grant W911NF-10-1-0246U. S. Army Research Laboratory and the U. S. Army Research Office grant W911NF-12-1-0273U. S. Army Research Laboratory and the U. S. Army Research Office grant 00101-80683U. S. Army Research Laboratory and the U. S. Army Research Office grant W911NF-10-1-0246U. S. Army Research Laboratory and the U. S. Army Research Office grant W911NF-12-1-027